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LANGUAGE 
E TINGS 
0 CONFERENCES 


March  3-6.  Convention  of  Teachers  of  English 
to  Speakers  of  Other  Languages,  10th. 

New  York,  NY. 

March  14-15.  North  American  Conference  on 
Afro-Asiatic  Linguistics,  4th,  and 
American  Oriental  Society.  Philadelphia, 
RA. 

March  21-24.  Association  for  Asian  Studies. 
/Washington,  D.C. 

^ March  25-27.  Conference  on  Composition  and 
y Communication.  Philadelphia,  PA. 

April  1-4.  Northeast  Conference  on  the 
Teaching  of  Foreign  Languages.  New  Yorfc, 
NY. 


April  3-5.  Annual  Meeting  of  the  Linguistics 
Association  of  Great  Britain,  Edinburgh, 
Scotland.  (Write:  J.  Christie,  Dept 
Linguistics,  15,  Buccleuch  PL,  Edinburgh 
EH8  9LN,  Scotland). 

April  5-9.  International  Conference  of 
Nordic  and  General  Linguistics,  3rd. 

Austin,  TX. 

April  6-9.  Acoustical  Society  of  America. 

^ Washington,  D.C. 

April  9-10.  College  English  Association. 
Cincinnati,  OH. 

April  15.  [Deadline  for  abstracts  for  LSA 
Sumner  Meeting]. 

April  15-17.  Conference  on  African  Linguis- 
tics, 7th.  Gainesville,  FL.  (Write: 

P.A.  Kotey,  Ctr  African  Studies,  470  LGH, 
Univ  Florida,  Gainesville,  FL  32611). 

April  19-23.  American  Educational  Research 
Association.  Las  Vegas,  NV. 

April  20-24.  Congress  of  the  International 
Association  for  the  Study  of  Italian 
Language  and  Literature,  9th.  Palermo, 
Italy.  (Write:  Robert  J.  Clements, 

AISLLI,  Rm  701,  Main  Bldg,  New  York  Univ, 
Washington  Sq,  New  York,  NY  10003). 

April  22-24.  University  of  Kentucky  Foreign 
Language  Conference.  Lexington,  KY„ 

April  22-24.  Southwest  Areal  Language  and 
Linguistics  Workshop,  5th.  San  Antonio, 

TX.  (Write:  B.L.  Hoffer,  Dept  English, 

Trinity  Univ,  San  Antonio,  TX  78284). 

April  22-25.  Chicago  Linguistic  Society. 
Chicago,  IL.  (Write:  Chicago  Linguistic 
Society,  Goodspeed  205,  1050  E 59th  St, 
Chicago,  IL  60637). 

May  6-8.  Conference  on  Perspectives  on 
Language.  Louisville,  ICY. 

June  21-26.  Conference  on  the  Psychology  of 
Language.  Stirling,  Scotland. 

June  2B-July  2.  International  Conference 
on  Computational  Linguistics.  Ottawa, 

Ont,  Canada. 

July  26-31.  Philippine-American  Communication 
Conference,  1st.  Manila,  The  Philippines. 
August  17-19.  World  Congress  of  the  Inter- 
national Reading  Association,  6th, 
Singapore. 

August  26-31.  World  Congress  of  Phoneticians, 
3rd.  Tokyo,  Japan. 

August  28-30.  European  Linguistics  Society. 
Salzburg,  Austria.  (Write:  G.  Drachman, 

Institut  Fflr  Sprachwissenschaft  der 
Universitat,  Imbergstrasse  2/III,  A-5020 
Salzburg,  Austria). 

September  1-4.  International  Phonology 
I1  (UNCLASSIFIED)  Meeting,  3rd.  Vienna,  Austria. 
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GENS  Joint  Mechanization  Gro\ 
conducted  the  first  tests  of 
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(CONFIDENTIAL 


' TIME’S  ’T^UNNING 
OUT! 


You'd  betteT 
hurry  if  you 
want  to  enter 
the  CMI  ESSAY 
CONTEST  or  the 
CLA  ESSAY  CON- 
TEST this  year! 

The  deadline 
for  submitting 
entries  to  the 
CMI  ESSAY  CON- 
TEST is  26  March 
and  the  deadline 
for  the  CLA  ES- 
SAY CONTEST  is 
19  March. 

For  complete  information,  see  The  NSA 
Technical  Journal , Vol.  XX,  No.  4,  Fall 
1975.  Or  call  the  following: 

CMI  Contest : f 


CLA  Contest: 


I 3957s 


or 
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PE  I^N  E Y 


Puzzle.  &an&,  mathematicians,  cryptanalysts'. 
Here's  an  opportunity  you  won't  want  to  m issl 
The.  Cryptomathematics  Institute.  (CMI)  has  a 
number  oi  copies  oi  PENNED  PUZZLES,  a collec- 
tion oi  6i  puzzler  by  CM  I'*  executive  direc- 
tor, Walter  Penney.  While  the  supply  lasts, 
a iKte  copy  oi  this  fascinating  uncl^l^led*  , ) 
book  will  be  tent  to  everyone  joining  xwl.  ' 

To  join,  6 end  your  name,  social  securtty^tum-®-3® 
ber,  organization,  telephone  number,  and  $ 3.00 
&or  1976  dues  to: f | CMI  Treasurer, 

S03.  
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An  Objective  Approach  to 

SCORING  TRANSLATIONS 


|Reprinted  from  QRL  ( Quarterly  Review 
for  Linguists),  November  1973 


Author's  note:  The  philosophy  underlying 
the  translation  grading  system  described  in 
this  paper  has  been  developed  and  applied  by 
Emery  Tetrault  and  myself*  with  many  valuable 
suggestions  from  our  colleagues  on  Profes- 
sional Qualification  Examination  ( PQE ) Com- 
mittees and  from  other  Agency  linguists . My 
use  of  the  pronoun  "we”  re fleets  this  collabo- 
ration. I personally  take  full  responsibility  , 
for  presenting  our  findings  here . 

* 

Translation  as  an  intellectual  activity  has 
been  practiced  since  antiquity  for  practical  as 
well  as  esthetic  reasons,  but  even  today  we 
have  reached  no  consensus  on  whether  it  is  even 
possible  to  fully  "bring  over"  from  one  lan- 
guage to  another  the  contents  of  a fairly  pe- 
destrian piece  of  prose,  let  alone  fiction  or  ] 
poetry.  We  debate  the  merits  of  machine  trans- 
lation as  a precursor  to  a truly  "scientific" 
discipline  or  argue  whether  a "free"  transla- 
tion of  a Greek  lyric  poem  does  not  better 
catch  the  flavor  of  the  original  than  one  which 
adheres  rather  closely  to  the  syntax  of  the 
original  and  results  in  an  awkward  (i.e.  "lit- 
eral") rendering  in  the  target  language.  In 
short,  the  domain  of  translation  has  not  been 
really  determined.  Our  own  work  never  demands 
(fortunately)  the  highly  creative  product  of 
the  artist  translating  Homer,  Dante,  or  Pushkin 
but  it  does  require  something  more  than  a purely 
mechanical  approach;  good  translators  have  a flair 
for  couching  in  idiomatic  English  the  thought  con- 
tent of  the  source-language  passage.  Poor  trans- 
lators, on  the  other  hand,  seem  never  to  master 
this  complex  operation  regardless  of  how  well 
they  may  know  the  source  language  or  how  well 
they  may  write  English  when  not  translating. 

If  we  had  only  the  two  kinds  of  translators 
to  judge  we  could  sort  them  into  sheep  and 
goats  and  have  done  with  it.  The  fact  is  that 
most  aspiring  translators  are  neither  brilliant 
nor  hopeless,  but  fall  somewhere  in  between, 
and  when  professional  certification  is  at  stake 
we  need,  in  the  interest  of  justice  to  the  can- 
didate as  well  as  to  the  system,  reasonably 
objective  criteria  for  making  a final  decision 
on  P&6&  or  &cUJL. 

Lacking  such  criteria  in  our  early  attempts 
to  evaluate  translations,  we  ended  too  often 
with  intuitive  decisions  not  transferable  to 
similar  cases  involving  the  same  source  lan- 
guage; nor  was  it  even  possible  to  compare  in- 


tuitive judgments  across  language  boundaries, 
in  source  language-to-English  situations. 

Over  the  past  2 or  3 years  my  colleagues  and 
I have  developed  a way  to  score  translations 
which  may  obviate  this  problem  to  a large  ex- 
tent even  though  our  results  have  admittedly 
been  far  from  perfect  (total  "objectivity"  in 
grading  any  kind  of  connected  text  is  of  course 
impossible) . Our  first  large-scale  use  of  the 
system,  which  I will  describe  later,  was  with 
the  Russian  PQE.  We  have  subsequently  tried  it 
in  a number  of  other  PQEs  involving  several 
languages,  mainly  Indo-European,  but  also  from 
other  families.  The  results  have  been  encour- 
aging enough  in  both  instances  for  us  to  recom- 
mend its  use  in  the  PQE  Handbook. 

Subjective  Approaches  to  Evaluating  Translations 

One  approach  to  scoring  translations  is  in  a 
sense  historical  in  that  the  evaluator  tries  to 
guess  the  reason  for  the  translator's  mistake  and 
in  so  doing  draws  unwarranted  inferences.  This 
attempt  may  be  useful  when  the  time  comes  to 
counsel  the  failing  or  borderline  aspirant  and 
the  problems  can  be  talked  out.  Unfortunately, 
though,  the  history  of  an  error  is  irrelevant  to 
the  rendering  of  a particular  text  and  whether 
the  aspirant  should  have  learned  a particular 
grammar  rule  in  a course  is  beside  the  point. 

A second  method  in  wide  use  is  to  decide  ar- 
bitrarily that  a certain  kind  of  mistake  (for 
example,  failure  to  recognize  proper  nouns  and 
inappropriate  translation  of  them)  is  sufficient 
cause  to  fail  a paper.  This  syndrome  is  evident 
among  longtime  subject-matter  experts  or  special- 
ists in  some  area  of  the  world  who  see  examina- 
tions solely  in  terms  of  their  immediate  inter- 
ests. 

One  example  of  each  approach  should  suffice. 
In  the  first  case,  a would-be  translator  missed 
a trickly  negative  conditional  construction  in 
the  first  sentence  of  a fairly  long  paragraph 
which  contained  other  negative  and  conditional 
elements  much  less  difficult.  In  the  interest  of 
coherent  discourse  the  translator  overrode  the 
signals  which  "he  should  have  learned"  and  pro- 
duced a sensible  but  wrong  text.  A second  trans- 
lator decided  that  the  name  of  a group  of  is- 
lands, which  historically  means  "fishing  plat- 
form," should  be  so  translated.  Not  only  did 
he  fail  to  recognize  the  proper  noun,  but  he  also 
saw  the  context  of  an  entire  paragraph  in  terms 
of  the  translation,  and  produced  a peculiar 
text  in  English. 

The  evaluators  of  the  first  passage  initial- 
ly decided  that  the  paper  should  fail  because 
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the  aspirant  did  not  know  enough  of  the  grammar 
of  his  language,  regardless  of  his  performance 
in  the  rest  of  the  examination.  (Fortunately, 
we  reconsidered  the  translation  as  a whole  and 
gave  the  paper  a marginal  pass.)  The  second 
faux  pas  made  in  an  early  PQE  apparently  caused 
such  anguish  to  the  evaluators  that  they  failed 
the  paper  mainly  on  this  point. 

I hasten  to  add  that  none  of  the  above  is 
meant  in  a caviling  spirit.  Translations,  wher- 
ever and  by  whomever  done,  have  ordinarily  been 
assessed  by  these  or  other  equally  subjective 
methods.  What  we  propose  here  is  an  attempt  to 
bring  a measure  of  uniformity  to  grading  trans- 
lations so  that  intuition  and  idiosyncrasy  play 
a minor  role  and  some  kind  of  objective  standard 
is  established.  The  framework  and  detail  of  the 
proposed  system  is  described  below. 

Linguistics  and  Evaluation  of  Translation 

Before  discussing  precise  error  weighting  I 
should  say  a few  words  about  linguistics,  dreary 
as  the  prospect  may  be,  and  follow  with  some 
comments  on  how  committees  apply  linguistic 
principles  in  test  evaluation. 

The  first  point  to  be  made  is  that  the  grad- 
ing system  to  be  described  is  based  insofar  as 
possible  on  a somewhat  artificial  distinction 
between  syntax  and  6 manticM . I will  therefore 
define  these  terms  insofar  as  they  apply  to  the 
present  case. 

• Syntax  is  the  system  of  relating  linguistic 
elements  to  one  another  in  phrase,  clause, 
or  sentence,  a process  which  is  concerned 
with  selecting  appropriate  grammatical 
forms  (hereafter  called  "affixes"),  posi- 
tions for  vocabulary  items,  or  a combina- 
tion of  both,  depending  upon  the  particu- 
lar language.  Syntactic  analysis  leads 
from  particular  texts  to  general  statements 
about  the  linguist  elements  of  a language. 
The  traditional  parsed  sentence  is  an  in- 
ductive exercise  proceeding  from,  for  ex- 
ample, the  sentence  "The  quick  brown  fox 
jumped  over  the  lazy  dog " to  a set  of  names 
or  symbols : Definite  article  + adjective  + ad- 
jective + noun  + verb  + past  tense  + preposi- 
tion + definite  article  + adjective  + noun. 

We  cannot,  however,  recapture  this  gem  by 
simply  reversing  the  procedure,  because, 
except  for  the  definite  article,  a possibly 
infinite  number  of  combinations  could  ap- 
pear. Nonetheless,  a command  of  the  syn- 
tax of  a foreign  language,  which  will  in- 
evitably differ  from  that  of  English  at 
many  points,  is  the  first  element  of  the 
language  the  student  necessarily  learns. 

• Semantics,  on  the  other  hand,  is  concerned 
with  particular  meanings  of  linguistic  ele- 
ments in  context.  To  the  extent  that  pat- 
terns of  feeling  and  thinking  between  lan- 
guages are  similar,  as  to  a large  extent 
they  will  be  in  the  Indo-European  languages. 


transfer  of  ideas  and  impressions  will  be 
on  the  whole  fairly  simple.  (Some  kinds  of 
texts  are  dependent  on  an  intimate  know- 
ledge of  a vastly  different  culture  sub- 
system, religious,  political,  and  so  on. 
These  texts  are  never  used  for  PQEs.)  The 
student  needs  a vocabulary  of  a couple  of 
thousand  words,  the  main  carriers  of  ideas, 
before  he  can  work  efficiently,  regardless 
of  how  good  his  syntactic  control  may  be. 

Obviously,  many  members  of  PQE  Committees 
have  not  had  much  formal  linguistic  training  in 
syntactic  and  semantic  analysis,  excellent 
though  they  may  be  in  the  language  they  are  con- 
cerned with.  Just  as  obviously,  there  is  no 
time  for  us  to  give  extensive  training  in  these 
subjects  to  people  who  are  busy  with  other  things. 
However,  a certain  minimum  of  linguistic  think- 
ing is  needed  to  make  our  system  work  and  we 
have  tried  to  impart  this  informally  to  commit- 
tees before  they  select  passages  for  translation 
and  evaluate  them.  Once  a committee  has  a gen- 
eral grasp  of  the  system,  the  members  should  be 
better  able  to  select  passages  containing  inter- 
esting syntactic  and  semantic  problems.  After 
having  produced  translations  of  passages  against 
which  papers  will  be  judged,  the  committee  can 
then  to  some  extent  predict  likely  errors.  When 
the  papers  are  actually  graded,  each  evaluator 
works  independently  to  derive  a score  based  on 
the  weighting  described  below  then  compared  his 
results  with  the  others. 

Some  differences  in  weighting  are  bound  to 
occur  and  may  be  irreconcilable,  but  generally 
the  findings  show  a narrow  spread  of  5 to  not 
more  than  10  points.  The  care  required  in  such 
careful  grading  insures  that  at  the  very  least 
major  errors  are  not  overlooked,  a result  not 
always  achieved  by  subjective  grading. 

Our  weighting  of  linguistic  elements  is  based' 
on  the  assumption  that  aspirants  should  be  ex- 
pected to  control  their  source- language  syntax. 

A failure  to  understand  relations  within  and 
among  sentences,  which  are  marked  by  position 
and/or  affixes  as  noted  above,  can  seriously 
distort  the  meaning  of  an  entire  paragraph; 
hence,  we  take  either  8 or  4 points  off  per  er- 
ror, depending  on  its  severity.  If,  for  exam- 
ple, a sentence  should  have  been  translated 
"Dog  bites  man " but  comes  out  n Man  bites  dogj  " 
we  would  subtract  8 points  because  all  the  re- 
lationships are  wrong.  If  a sentence  which 
ought  to  have  been  translated  "We  stopped  near 
the  train"  is  rendered  "We  stopped  the  train>  " 
we  would  take  off  4 points  since  the  subject- 
verb  relationship  is  correct  even  though  the 
rest  of  the  sentence  is  wrong.  As  we  have  set 
70  as  a passing  score,  four  8-point  errors  will 
fail  a PQE. 

Certainly  the  examples  above  are  oversimpli- 
fications of  translation  problems  which  can  be 
knotty.  Our  experience  has  shown,  however,  that 
in  9 out  of  10  cases  committees  can  achieve  con- 
sensus on  the  kind  and  degree  of  syntactic  error 
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not  only  for  Russian  examinees  but  also  for  as- 
pirants in  Indo-European  and  Semitic  languages 
generally.  Where  we  have  had  problems,  it  has 
been  in  languages  such  as  Chinese  or  Vietnamese 
where  the  intersection  of  "syntax"  and  "semantics" 
has  not  begun  to  be  explored,  let  alone  taught 
to  unversity  or  DLI  students.  If  aspirant  trans- 
lations from  these  languages  go  far  afield  and 
reverse  the  meaning  of  the  original,  or,  even 
worse,  do  not  even  fall  in  the  correct  subject- 
matter  area,  the  point  system  above  is  still  ap- 
plied: wrong  is  wrong.  It  would  of  course  be 
possible  to  set  as  passing  a score  lower. than  70 
for  languages  structurally  removed  from  English, 
but  a decision  to  do  so  would  be  made  on  grounds 
other  than  linguistics. 

Semantic  errors  are  "charged  at  a lower  rate" 
(2  points  per  error) , not  because  we  regard  vo- 
cabulary as  unimportant  — far  from  it.  As  I 
noted  above,  a knowledge  of  syntax  alone  won't 
do,  for  the  idea  structure  mainly  depends  upon 
lexical  units.  However,  our  experience  has  shown' 
that  students  of  foreign  languages  pick  up  vo- 
cabulary unevenly,  probably  because  no  systematic 
attempt  to  teach  it  has  ever  been  devised  and 
successfully  applied.  Nor,  for  that  matter,  are 
students  properly  instructed  in  the  use  of  either 
monolingual  or  bilingual  dictionaries.  To  the 
extent  that  vocabulary  instruction  is  given  at 
all,  it  is  given  on  the  job  and  takes  the  form 
of  ad  hoc  corrections  and  suggestions  on  how  to 
use  one  or  another  dictionary  to  better  effect. 
This  process  may  be  tedious  but,  provided  the 
subject  matter  translated  is  not  abstruse,  is 
easy  in  comparison  with  mastering  a new  syntax. 

For  these  reasons  it  seemed  best  for  us  to 
accommodate  to  the  existing  situation  and  apply 
the  lighter  penalty.  Even  so,  we  still  encounter 
situations  in  which  it  is  almost  impossible  to 
decide  whether  an  error  is  in  fact  best  viewed 
as  semantic  or  syntactic.  If  a source- language 
pronominal  reference  is  translated  by  the  wrong 
pronoun,  is  this  a simple  error  at  a given  point 
(hereafter,  "punctil inear  error")  or  could  it 
open  the  way  for  misunderstanding  several  sen- 
tences (syntactic  error)?  Contextual  judgment 
is  the  only  method  for  making  such  a decision. 

On  the  other  hand,  some  apparently  pure  grammar 
markers,  such  as  tense  forms,  can  be  mistrans- 
lated without  in  any  way  distorting  the  basic 
noun-verb  relationships.  Thus,  " the  dog  bites 
the  man"  and  "the  dog  bit  the  man"  have  the  same 
noun-verb-noun  arrangement:  the  difference  lies 
in  the  time  of  the  event  identified  at  a given 
point  in  the  text.  A tense  error  is  hence  punc- 
tilinear  and  costs  2 points. 

As  I have  suggested,  it  is  often  difficult 
for  PQE  Committees  to  reach  a consensus  at 
every  point  of  semantic  and  syntactic  error  in 
translation  from  the  source  language.  The  task 
of  evaluating  the  translation  from  the  point  of 
view  of  English  usage  and  violation  of  conven- 
tion is  even  harder,  but  extremely  awkward  Eng- 
lish does  have  an  overall  effect  of  blurring 


meaning  or,  at  the  very  least,  causing  the 
reader  unnecessary  strain.  Violation  of  accep- 
table English  we  call  "poor  cooccurrence." 

When  the  members  of  a committee  are  unani- 
mous in  judging  an  English  phrase  to  reflect 
poor  cooccurrence,  one  point  is  assessed.  Here 
are  examples  of  the  kind  of  poor  usage  we  have 
met:  "an  oblivious  scholar  to  practical  matters" 
instead  of  "a  scholar  oblivious  to.  . ."  (awk- 
ward splitting  of  adjective  and  accompanying 
preposition);  "the  car  collided  against  the 
bus"  instead  of  ".  . . with  the  bus"  (a  seem- 
ingly unlikely  mistake  for  a native  speaker  of 
English  to  make,  but  this  type  is  of  high  fre- 
quency nonetheless) ; "Foreign  Minister  Rogers" 
instead  of  "Secretary  of  State  Rogers"  (a  vio- 
lation of  conventional  usage) . 

Hundreds,  if  not  thousands,  of  similar  pec- 
cadilloes have  been  and  are  being  committed  un- 
der the  present  system  and  are  the  object  of 
heated  remonstrances  from  the  failing  aspirant. 
("Why  do  they  have  to  nitpick  everything?")  It 
has  been  our  personal  experience  that  commit- 
tees on  the  whole  err  on  the  side  of  generosity 
in  cases  of  one-point  errors,  and  overlook  al- 
together misspellings  and  wrong  punctuation 
where  these  do  not  cause  major  problems  in  un- 
derstanding the  English  text.  Even  those  eval- 
uators who  take  violent  exception  to  misspelling 
will  not  fail  a paper  for  this  alone. 

* 

To  sum  up,  it  has  been  our  intention  to  es- 
tablish a system  of  grading  which  will  be  as 
objective  as  possible.  Using  a base  of  100  and 
with  70  as  passing,  we  have  specified  3 kinds 
of  errors.  The  first  2,  syntactic  and  semantic, 
concern  command  of  the  source  language;  the 
third  bears  upon  English  usage.  Syntactic 
errors  are  of  two  orders  of  severity  reflected 
in  the  8-  and  4-point  penalties,  while  semantic 
mistakes  are  charged  2 points  and  English  mis- 
use, or  poor  cooccurrence,  costs  one. 

There  are,  to  be  sure,  many  doubting  Thomas- 
es who  question  the  efficiency  of  the  system, 
either  because  they  feel  it  is  too  complex  to 
master  or  that  the  time  spent  in  mastering  it 
might  better  be  devoted  to  something  else  and 
that  the  intuitive  or  "seat-of-the-pants"  method 
will  do  as  well.  While  conceding  the  time  re- 
quired to  be  considerable,  the  final  result,  we 
feel,  more  than  justifies  the  effort  and,  in  the 
absence  of  a better  system,  ought  to  prevail. 


Editor's  note:  The  system  described 
in  the  reprinted  article  has  been  in  use 
at  the  Agency  since  July  1972 . | I p 

is  currently  writing  a paper  in  wnvcn  ne 
will  summarize  the  experience  that  the 
PQE  Committees  have  had  in  using  the  sys- 
tem and  will  suggest  theoretical  and 
practical  improvements . 
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How  does  an  NSA  language  professional  stack 
up  against  language  professionals  elsewhere? 

That  question  cannot  really  be  answered  inas- 
much as  apples  and  pineapples  are  not  directly 
comparable.  An  insight  at  the  very  lowest  level* 
of  professionalization  can  be  gained,  however,  by 
comparing  the  criteria  for  NSA's  open-source 
translation  (Part  IA  of  the  Professional  Quali- 
fication Examination)  and  those  for  accreditation 
by  the  American  Translators  Association  (ATA) . 

NSA 's  test 

The  NSA  language  test  is  of  such  length  in  the 
foreign  language  as  to  generate  a total  of  450  to 
600  words  of  English  translation.  Time  is  not  a 
factor,  therefore  examinees  can  take  as  long  as 
4 hours  for  this  section  of  the  PQE.  Texts  do 
not  pose  any  problems  of  content  substance;  that 
is,  they  do  not  require  an  overly  specialized 
knowledge  of  subject  matter.  They  are  usually 
articles  by  professional  journalists  on  a topic 
of  interest  to  the  general  public  --at  the  dif- 
ficulty level  of  Reader's  Digest  or  a weekly 
news  periodical.  Any  lexical  item  in  the  text 
which  is  not  in  the  standard  bilingual  diction- 
ary is  glossed  in  a footnote. 

NSA  graders  use  a pointe-off  system  in  grad- 
ing the  test.  They  give  more  relative  weight 
to  errors  involving  the  logical  relationships 
among  major  sentence  parts  and  to  identifying 
specific  items  in  the  source  text  (that  is, 
markers  for  number,  case,  gender,  tense,  aspect, 
mood,  voice,  etc.).  Each  sentence  is  graded  by 
itself;  no  more  than  8 points  is  taken  off  for 
any  one  sentence.  If  the  logical  subject-verb- 
object/complement  relationship  in  the  translation 
is  not  substantially  the  same  as  that  of  the 
source- language  sentence,  no  fewer  than  4 and  no 
more  than  8 points  are  taken  off.  Experience  in 
various  test  committees  has  shown  that  examinees 
who  get  the  maximum  of  8 points  taken  off  on 
each  of  3 sentences  seldom  pass  the  test. 


For  each  translation  error  involving  a 
single  word  (content  words  such  as  nouns, 
adjectives,  verb  stems;  function  words  such 
as  prepositions  and  conjunctions,  and  gram- 
matical affixes)  2 points  are  deducted  -- 
up  to  a maximum  of  8 points  in  any  one 
sentence.  For  errors  involving  poor  selec- 
tion of  dictionary  meanings  of  a lexical 
item  or  poor  word  collocations  (words  that 
just  don't  fit  together  right  in  English), 
only  one  point  is  deducted. 

From  the  foregoing  explanation,  it  can  be 
seen  that  a sentence  in  the  English  translation 
might  contain  one  major  error  involving  logical 
relationships  (say,  8 points) , 3 single-word 
errors  (6  points),  and  perhaps  2 collocation  or 
word-choice  errors  (2  points) , but  the  NSA 
gTader  would  not  deduct  16  points  for  that  sen- 
tence --  instead  he  would  deduct  only  the  maxi- 
mum of  8 points  per  sentence. 

If  the  translation  of  Part  IAof  the  Agency's 
PQE  has  fewer  than  30  points  deducted,  it  is 
considered  to  be  a "pass."  But  a translation 
that  just  passes  is  far  from  good.  It  is  not  as 
simple  as  saying  that  a translation  with  30 
points  deducted  is  "30%  inaccurate."  Instead, 
such  a translation  may  be  "all  wrong,"  because 
it  will  almost  certainly  contain  several  errors 
of  fact,  any  one  of  which  could  be  critical  in 
a real-life  translation  situation.  So,  if 
such  a poor  examinee  were  to  be  certified  as  a 
linguist  and  were  to  produce,  on  an  everyday 
basis,  operational  translations  containing 
such  critical  errors  of  fact,  one  can  only 
imagine  the  tremendous  amount  of  work  and  re- 
sponsibility that  would  be  placed  on  the  lan- 
guage checker  who  would  have  to  correct  the 
translations  before  issuing  them.  Or  on6  could 
imagine  the  inherent  dangers  if  such  transla- 
tions, in  a stress  situation,  were  issued  to 
the  customer  without  being  corrected. 
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If  a particular  grader  deducts  between  30 
and  40  points,  the  translation  must  be  debated 
in  committee;  if  the  majority  of  the  committee 
members  feel  that  enough  of  the  minor  errors 
would  have  been  caught  and  corrected  in  the 
checking,  proofreading,  and  analysis  phases, 
a translation  with  as  many  as  40  points  might 
actually  be  allowed  to  pass.  PQE  committee 
members  sometimes  lean  over  backwards  in  at- 
tempting to  pass  certain  marginal  papers.  They 
claim  that  if  only  a few  examinees  pass,  man- 
agement tends  to  say  that  the  test  is  too  hard, 
never  that  the  applicants  are  just  not  certifi- 
able as  professional  linguists. 

One  thing  must  be  borne  in  mind,  however. 

A person  who  passes  the  PQE,  even  with  distinc- 
tion, is  only  certified  as  a linguist;  it  still 
might  take  him  many  years  of  on-the-job  experi- 
ence to  become  a senior  linguist  (something 
that  a person  becomes  without  having  a piece 
of  paper  to  prove  it).  Dr.  Jaffe  used  to  com- 
pare this  situation  with  that  of  a medical  stu- 
dent being  awarded  a diploma  from  medical 
school  and  the  right  to  apply  for  a license  to 
practice  medicine;  he  would  still  need  years  of 
experience  to  become  an  expert  in  his  field.  If, 
then,  a person  who  passes  a PQE  without  debate 
has  demonstrated  the  potential  to  become  a 
senior  linguist,  the  person  who  is  allowed, 
after  debate,  to  squeak  through  a PQE  has  de- 
monstrated, if  anything,  the  ability  to  function 
at  only  the  very  lowest  level  of  linguistic 
competency. 

ATA  test 

The  American  Translators  Association  is  a 
different  animal.  Its  members  include  both 
literary  translators  and  technical  translators. 
Literary  translation  is  a creative  art,  and  is 
not  tested.  But  since  all  translators  make 
most  of  their  money  from  translating  technical , 
scientific,  and  legal  material,  technical  trans- 
lation is  used  for  ATA  accreditation.  The 
length  of  the  ATA  test  is' approximately  750 
words.  Time  is  a factor;  precisely  3 hours  is 
allotted  for  the  test.  As  with  the  NSA  text, 
the  texts  do  not  pose  any  problems  of  content 
substance:  lexical  items  not  in  the  standard 
bilingual  dictionary  are  glossed  in  footnotes. 
Selections  tend  to  reflect  the  subject  matter 
of  professional  translators  (international 
meetings,  symposia,  national  projects,  medical 
research,  technology,  science,  etc.),  but  in  a 
popularized  version  at  the  newspaper  or  Reader1  s 
Digest  difficulty  level. 

Backgrounds  and  specializations  of  the  ATA 
members  vary  greatly.  The  examinees  are  ac- 
cordingly offered  five  selections  in  five  some- 
what different  subject  fields,  from  which  they 
choose  three  to  translate.  The  topics  in  the 
test  that  I recently  took  were  the  Apollo -Soyuz 
flight,  the  Helsinki  Conference,  blood  immunolo- 
gy, a legal  case  involving  patents,  and  the 
prospective  Canadian  Health  Service  System. 


These  selections  do  not  reflect  the  diffi- 
culty level  of  the  foreign- language  texts  that 
usually  confront  professional  translators.  But 
the  grading  criteria  do  reflect  the  requirement 
of  the  customers  who  let  contracts  for  trans- 
lators. The  ATA  examinees  are  told  to  be  pre- 
cise and  fairly  literal , and  to  be  extremely 
carefuly  in  paraphrasing  so  as  not  to  lose  one 
iota  of  the  sense  of  the  clause  or  sentence. 
They  are  warned  that  omissions  of  words  (except 
for  such  low- information  words  as  schon3  dock, 
und  zwar , en  effets  znachit ) can  be  extremely 
costly  and  can  even  result  in  failing  a selec- 
tion. A candidate  must  pass  two  out  of  the 
three  passages  selected.  Only  one  serious  er- 
ror in  a passage  is  permitted;  two  serious  er- 
rors constitute  a fail  for  that  selection  (of 
course,  a large  number  of  nonserious  errors 
could  also  constitute  a fail). 

Following  is  a passage  from  a statistical 
text,  Calcul  des  probabilities  that  was  used  in 
the  French  test  administered  at  the  ATA  work- 
shop in  California  in  the  summer  of  1975.  The 
passage  is  followed,  in  turn,  by  three  of  the 
translations  done  by  examinees  (for  the  con- 
venience of  the  grading  committee,  the  transla- 
tions are  typed  prior  to  grading) . The  under- 
lined words  and  the  notations  in  the  left-hand 
margin  were  made  by  one  of  the  ATA  graders  (a 
capital  E is  the  symbol  for  a major  error;  a 
lower-case  e is  the  symbol  for  a minor  error; 

* mistr . stands  for  mistranslation.) 

Comparison  of  ATA  and  NSA  grading  systems 

To  compare  the  ATA  and  NSA  grading  systems, 
the  typed  translations,  including  the  ATA  grad- 
ers underlines  and  notations,  were  shown  to  an 
NSA  linguist  with  experience  in  grading  French 
PQEs.  He  made  his  own  underlines  (======) > 

most  of  which  coincided  with  errors  noted  by 
the  ATA  grader,  and  indicated  in  the  right-hand 
margin  the  number  of  points  that  would  be  de- 
ducted in  accordance  with  the  Agency* s PQE 
criteria.  For  example,  in  translation  #1 
the  ATA  grader  considered  the  error  "for  both" 
(mistranslation  of  pour  les  dieuxj  "for  the 
gods,"  misread  as  pour  les  deux)  to  be  a major 
error,  whereas  the  NSA  grader  deducted  2 points 
for  it. 

Since  the  Agency* s PQEs  are  usually  twice  as 
long  as  the  ATA  French  selection,  let  us  assume 
that  the  translators  would  have  had  an  equal 
error  rate  for  an  additional  300  words.  In  that 
event,  the  results  would  have  been  as  follows: 


NSA  grade 


Trans- 

lator 

ATA 

grade 

Raw 

score 

Extrapolated 

score 

Pass/ 

fail 

1 

Fail 

84 

68 

Marginal 

2 

Fail 

72 

44 

Fail 

3 

Pass 

100 

100 

Pass 
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I.  LE  HASARD 

"Comment  oser  parler  des  lois  du  hasard? 
Le  hasard  n'est-il  pas  l'antithese  de  toute 
loi?"  Ainsi  s'exprirae  Bertrand,  au  debut 
de  son  Calcul  de8  probab'ilit&s . La  probabi- 
lity est  opposee  a la  certitude;  c'est  done 
ce  qu'on  ignore  et,  par  consequent  semble- 
t-il,  ce  qu'on  ne  saurait  calculer.  II  y a 
l£  une  contradiction  au  mo ins  appa rente  et 
sur  laquelle  on  a deja  beaucoup  ecrit. 

Et  d'abord  qu'est-ce  que  le  hasard?  Les 
anciens  distinguaient  les  phenomenes  qui 
semblaient  obeir  a des  lois  harmonieuses , 
etablies  une  fois  pour  toutes,  et  ceux  gu1  ils 
attribuaient  au  hasard;  e'etaient  ceux  qu'on 
ne  pouvait  prevoir  parce  qu'ils  etaient 
rebelles  a toute  loi*  Dans  chaque  domaine 
les  lois  precises  ne  decidaient  pas  de  tout, 
elles  tragaient  seulement  les  limites  entre 
lesquelles  il  etait  permis  au  hasard  de  se 
mouvoir.  Dans  cette  conception,  le  mot 
hasard  avait  un  sens  precis,  ob jectif:  ce 
qui  etait  hasard  pour  l'un,  etait  aussi 
hasard  pour  1* autre  et  meme  pour  les  dieux. 

Mais  cette  conception  n'est  plus  la 
notre;  nous  sommes  devenus  des  deterministes 
absolus,  et  ceux  memes  qui  veulent  re server 
les  droits  du  libre  arbitre  humain  laissent 
du  moins  le  determinisme  regner  sans  partage 
dans  le  monde  inorganique.  Tout  phenomene, 
si  minime  qu'il  soit,  a une  cause,  et  un 
esprit  infiniment  puissant,  infiniment  bien 
informe  des  lois  de  la  nature,  aurait  pu  le 
prevoir  des  le  commencement  des  siecles.  Si 
un  pareil  esprit  existait,  on  ne  pourrait 
jouer  avec  lui  a aucun  jeu  de  hasard,  on 
perdrait  toujours. 

Pour  lui,  en  effet,  le  mot  de  hasard 
n'aurait  pas  de  sens,  ou  plutot  il  n'y 
aurait  pas  de  hasard.  C'est  a cause  de 
notre  faiblesse  et  de  notre  ignorance  qu'il 
y en  aurait  un  pour  nous.  Et,  meme  sans 
sortir  de  notre  faible  humanite,  ce  qui  est 
hasard  pour  lr ignorant,  n'est  plus  hasard 
pour  le  savant.  Le  hasard  n'est  que  le 
mesure  de  notre  ignorance.  Les  phenomenes 
fortuits  sont,  par  definition,  ceux  dont 
nous  ignorons  les  lois. 


I.  CHANCE 

"How dare wespeakof  the  laws  of  f\\ 

chance?  Is  not  cKance  the  antithesis  of  \J 
every  law?"  Thus  Bertrand  expresses  him- 
self, at  the  beginning  of  his  "Calculation 
of  Probabilities . " Probability  is  op- 
^ . posed  to  certainty;  thus  it  is  what  is  un- 
£ known,  /S  what  we  would  not  know  how  to 

^ calculate.  There  is  at  least  an  apparent 

contradiction  here,  one  on  which  a great 
deal  has  already  been  written. 

- First  off,  what  is  chance?  The  ancients 

distinguished  between  t)^  phenomena 

°n 

which  seemed  to  obey  harmonious  laws , es- 
tablished once  for  all,  and  those  which 
they  attributed  to  chance;  they  were  the 
ones:  that  couldn' t be  anticipated  because 
^ ::they  were  unamenable  to  all  law.  In  each 
domain,  no  precise  laws  were  determined; 

C \e  they  traced  only  the  boundaries  among 
which  chance  was  allowed  to  operate.  In 
this  concept,  the  word  "chance"  had  a pre- 
cise, objective  meaning:  what  was  chance 
£ for  one  was  also  chance  for  the  other,  and 

even  for  both.  ^ (^/ 

However,  this  is  not  our  way  of  looking*^ 
at  it_£.  we  have  become  absolute  determin-^ 
ists , and  even  the  Same  ones  Who  want  to  g 
^ save  a place  for  free  will  in  mankin^O 
* never  the  le  samlet  determinism  hold  un^ 
trammeled  swSy  in  the  inorganic  world. 

Every  phenomenon,  no  matter  how  minute  it 
may  be,  has  a cause,  and  an  infinitely  pow- 
erful mind,  infinitely  well  informed  of  the 
laws  of  nature,  would  have  been  able  to  fore- 
tell it  since  the  beginning  of  time.  If  such 
^ a mind  existed,  one  would  not  be  able  to  play 
pj**-'1  1 any  games  of  chance  with  ib{ \ — one  would 
’ always  lose.  \Z/ 

For  that  mind,  in  fact,  the  word  "chance" 
would  have  no  meaning,  or  rather  there  would 
be  no  chance.  It  is  because  of  our  weakness 
and  our  ignorance  that  there  would  be  chance 
with  us.  Also,  even  without"gofng" outside^, 
o?=Htfr:?eeble  humanity,  what  is  chance  to  vi 
the  ignorant  is  no  longer  chance  to  the  wise 
man.  Chance  is  only  the  measure  of  our  ig-fr 
6 norance.  chance_phenomena , by  defi-  fa] 

nition,  are  those”whose  laws  we  do  not  know /“I 
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Translator  #3 


I. 


CHANCE 


I. 


CHANCE 


"How  does  one  dare  to  speak  of  the 
laws  of  chance?  Isn't  chance  the  antithe- 
sis of  all  law?"  Bertrand  thus  expresses 
himself,  at  the  beginning  of  his  "Calcula- 
tion of  Probabilities. " Probability  is 
opposed  to  certainty?  it  is  thus  what  one 
does  not  know  and,  consequently,  it  seems, 
what  one  does  not  know  how  to  calculate.  <3> 


* There  is  there  at  least  an  apparent  con-^J 
tradictionj  a%d  one  on  which  much  has 
already  been  written. 

A^4,  ^irst,  what  is  chance?  The  an-  CD 
eSl  singled^out  phemomena  that  seeme<^)| 

to  obey  harmonious,  firmly  established 
laws  and  those  they  attributed  to  chance? 

^TY- there  were  those  that  one  couldn'  t forsee^) 
bicaule~they  were  rebellious  to  all  Q 
law.  In  each  domain,  precise  laws  did  not 
decide  everything,  they  only  traced  the 
limits  between  which  chance  was  permitted 
to  operate.  In  this  conception,  the  word 
"chance"  had  a precise,  objective  meaning: 
that  which  was  chance  for  one,  was  also 
chance  for  the  other  and  even  for  the  gods . 

But  this  idea  is  no  longer  ours?  we  have 
become  absolute  determinists , and  even 
those  who  wish  to  reserve  the  laws  of  human 
free-will^Leavefja^T  leas^  tfr£  determin- 
ism  to  rule  without  division  in  the  in- 


e 

e 


JfV, ^organic:  world.  All  phenomenun£ffif._ my  1L> ^ 
as  small  as  it  is,  has  a cause,  and  an 
£ infinitely  powerful  spirit,  infinitely  Qt 
well  formed  to  the  laws~of~hature , would 


^ \\*  have  Been  aSle  to  forsee  it  from  the  be- 


V’  ginning  of  the  centuries. 


If  a like 


© 

spirit  existed,  no  game  of  chance  could 

be  played  with  it,  one  would  always  lose. 


For  it,  indeed,  the  word  of  chance 


. ==  © 
would  not  have  any  sense , or  soon  there  QfQ 

would  not  be  any  chance.  It  is  because 
of  our  weakness  and  our  ignorance  that 
there  would  be  one  for  us . And , even  (Q 
without  leaving  our  weak  humanity , 
that  which  is  chance  for  the  ignorant , is 
no  longer  chance  for  the  well-informed. 
Chance  is  only  the  measure  of  our  igno- 
^ ranee.  Fortuitous  phenomena  are,  by 

definition,  those  of  which  we  do  not  Vi/ 
know  the  laws.  =*====  ^ 

fcji 


"How  can  we  dare  to  speak  of  the 
laws  of  chance?  Is  not  chance  the  anti- 
thesis of  all  law?"  Thus  speaks  Bertrand, 
at  the  beginning  of  his  "Calculation  of 
Probabilities . " Probability  is  opposed 
to  certitude?  thus  it  is  what  we  do  not 
know  and  so,  apparently,  what  we  cannot 
calculate.  There  is  here  an  at  least 
apparent  contradiction  about  which  much 
has  already  been  written. 

First  of  all,  what  is  chance?  The 
ancients  distinguished  between  phenomena 
which  seemed  to  obey  harmonious,  defi- 
nitely established  laws  and  those  which 
they  attributed  to  chance.  The  latter 
were  phenomena  that  could  not  be  foreseen 
because  they  were  contrary  to  all  law. 

In  each  field  the  precise  laws  were  not 
all-determining?  they  merely  defined  the 
limits  within  which  chance  was  permitted 
to  operate.  In  this  conception,  the 
word  "chance"  had  a precise,  objective 
meaning:  what  was  chance  for  one  man  was 
chance  for  another,  and  even  for  the  gods . 

But  this  conception  is  no  longer  ours? 
we  have  become  absolute  determinists,  and 
even  those  who  wish  to  reserve  the  rights 
of  human  free  will  at  least  permit  deter- 
minism to  reign  unchallenged  in  the  inor- 
ganic world.  Every  phenomenon,  however 
small,  has  a cause,  and  an  infinitely 
powerful  mind,  infinitely  well  informed 
of  the  laws  of  nature,  could  have  fore- 
seen it  from  the  beginning  of  time.  If 
such  a mind  existed,  we  could  -not  play 
any  game  of  chance  with  it;  we  should 
always  lose . 

For  this  mind,  in  fact,  the  word 
"chance"  would  have  no  meaning,  or  rather 
there  would  be  no  chance.  It  is  because 
of  our  weakness  and  our  ignorance  that 
one  may  exist  for  us.  Even  without  de- 
parting from  our  weak  humanity,  what  is 
chance  for  the  ignorant  man  is  no  longer 
chance  for  the  scientist.  Chance  is 
merely  the  measure  of  our  ignorance. 
Fortuitous  phenomena  are,  by  definition, 
those  whose  laws  we  do  not  know. 
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One  final  remark  about  NSA  PQEs  and  ATA's 
accreditation  examinations:  they  are  both  de- 
signed to  set  the  lowest  threshold  level  of 
professionalization,  no  more!  At  NSA  the  em- 
phasis is  placed  on  accuracy  of  content.  In 
the  ATA,  accent  is  placed  on  precise,  literal 
accuracy  and  on  speed.  A person  who  can  com- 
plete the  exam  in  exactly  3 hours  will  be  able 
to  make  a bare  living  just  above  the  poverty 
level  in  translating  material  of  that  difficul- 
ty, using  conventional  paper-and-pencil  tech- 
niques and  typing  up  the  finished  copy  himself. 
Translators  therefore  specialize  in  specific 
technical  fields  where  they  can  get  at  least 
double  or  triple  the  basic  translation  pay 
rate.  Then  too,  professional  translators  often 
dictate  their  translations  into  recording 
equipment  and  pay  a typist  to  transcribe  the 
translation  into  finished  copy;  despite  this 
additional  expense,  this  method  is  beneficial 
to  the  translator  since  it  enables  him  to  at 
least  double  his  translation  output.  Conse- 
quently, a free-lance  professional  translator, 
after  gaining  experience  and  one  or  more 
specialties,  should  be  able  to  earn  at  least 
$36,000  a year.  Even  a part-time  free-lance 
(moonlighter)  translator  can  earn  as  much  as 
$5000  a year.  These  figures  indicate  that  ATA 
accreditation  is  worth  quite  a bit  in  dollars 
and  cents.  They  also  indicate  that,  according 
to  ATA  standards,  the  majority  of  certified  NSA 
linguists  are  overpaid,  while  the  few  truly 
professional  certified  linguists  at  NSA  are 
underpaid. 
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SUBJECT:  Cryptolo<g  Article , Oct  75 

In  one  of  the  gaps  which  occurred  dur- 
ing the  holiday  season  I got  to  read  your 
"...Proud  and  Bitter  Memories"  article.  Your 
message  is  clear,  as  are  your  feelings  about 
the  people  you  worked  with  (and  for).  I 
think  you've  done  a service  for  those  of  us 
here  who  didn't  have  much  to  do  with  "the  . 
problem"  by  writing  that  article.  It  wasn't 
enjoyable  reading,  but  I'm  recommending  it 
to  all  in  W.  . . 


cc:  Ch,  B 

Ch,  PI  A 

Editor,  Cryptolog,  P16 


Communications  Security 
I t Establishment 


I very  much  enjoyed  the  October  issue  of 
Crypto log,  especially!  T "Proud 

and  Bitter  Memories"  and  the  extract  from 
his  "Uncertain  Origins"  which  you  put 
across  as  a splendid  Double-Crostic.  Not- 
ing that  you  are  obviously  very  enthusi- 
astic about  this  kind  of  puzzle,  I am  en- 
closing a copy  of  a related  type  which  I 
constructed  for  fun,  but  on  completion 
realized  was  not  economically  viable,  since 
it  is  much  more  difficult  and  time-consum- 
ing to  create  than  to  solve.  . . I don't 
think  I shall  ever  construct  another  one, 
so  this  is  probably  an  authentic  O'Neill 
hapax  legomenon . . . 

Kevin  O'Neill 


P.L.  86-36 


Readers  who  would  like  a copy  of 
Mr.  O’Neill's  puzzle  may  get  one 
by  calling  the  Editor  on  5642s . 

(UNCLASSIFIED) 
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